Fast Inverse Compositional Image Alignment with Missing Data and Re-weighting

Image alignment is often performed using variants of the Lucas-Kanade algorithm [3]. Among these variants, the inverse compositional (IC) [1] method and the efficient second-order minimization (ESM) [2] method are the most efficient variants. While ESM is computationally more efficient for 2D image alignment problems such as homographies, IC is more computationally efficient for image alignment problems using RGBD data where it is expensive to re-compute the Jacobian. In this paper, we look at methods to accelerate the convergence of IC and ESM algorithms while remaining robust to outliers. We propose a preconditioning strategy to perform inverse composition with reweighting and missing data which avoids the need to re-compute the Jacobian and its Hessian at every iteration. We also consider how the effects of image noise and/or spectral aliasing over the scale of the deformation can be used to speed up the convergence of all these methods. The inverse compositional Lucas-Kanade algorithm minimizes the following cost function: